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Abstract—The proper names of software engineering functions 
and methods can greatly assist developers in understanding and 
maintaining the code. Most researchers convert the method name 
generation task into the text summarization task. They take the 
token sequence and the abstract syntax tree (AST) of source code 
as input, and generate method names with a decoder. However, 
most proposed models learn semantic and structural features 
of the source code separately, resulting in poor performance 
in the method name generation task. Actually, each token in 
source code must have a corresponding node in its AST. Inspired 
by this observation, we propose SGMNG, a structure-guided 
method name generation model that learns the representation 
of two combined features. Additionally, we build a code graph 
called code relation graph (CRG) to describe the code structure 
clearly. CRG retains the structure of the AST of source code 
and contains data flows and control flows. SGMNG captures the 
semantic features of the code by encoding the token sequence 
and captures the structural features of the code by encoding the 
CRG. Then, SGMNG matches tokens in the sequence and nodes 
in the CRG to construct the combination of two features. We 
demonstrate the effectiveness of the proposed approach on the 
public dataset Java-Small with 700K samples, which indicates 
that our approach achieves significant improvement over the 
state-of-the-art baseline models in the ROUGE metric. 

Index Terms—method name generation, graph representation, 
code semantic features, code structural features 


I. INTRODUCTION 


Proper names of methods in software engineering are signif- 
icant in program development and maintenance. A descriptive 
method name illustrates the key functions of the method, and it 
helps developers have a general understanding of the program 
and avoid the misuse of methods. Each method in the program 
is made up of several simple methods, and their names are 
often included in the hump naming style. Conversely, improper 
method names often confuse developers’ understanding of 
methods and even lead to the misuse of program methods, 
causing great difficulties for program maintenance and up- 
grades [1]. Developers may not understand the semantics and 
structure of the code segment accurately when programming, 
so it is possible that inconsistent method names are assigned 
in the code writing process. In addition, during the process 
of program upgrades, some method entities have already been 
modified, but the method names have not been updated at the 
same time. In summary, the manually written method names 
are likely to be inconsistent with the method entity. [2]. 


* Corresponding author. 


Many researchers have adopted different approaches to 
suggest consistent method names according to method con- 
tents. For example, static rules are constructed for source 
code analysis to suggest method names [3]. Obviously, the 
effectiveness of these analysis-based approaches depends on 
the rules constructed, and these rules cannot apply to different 
programming languages. AST is a data structure used for 
static syntax analysis of source codes, and it can provide 
additional structural information of the source code. Therefore, 
many approaches recommend method names based on the 
structural similarity of ASTs [4] [5]. However, as Jiang et al. 
[6] mentioned, these methods have limitations in suggesting 
proper method names for developers. Because such methods 
are based on retrieval, its generalization ability is relatively 
weak when processing new samples. 

The above methods are all retrieval-based methods, and we 
can also generate method names through an encoder-decoder 
model. The sequence of method names can be obtained by 
decomposing method names according to the naming rules. 
Thus, the method name prediction problem can redefine as 
a sequence generation problem [7]. With the development 
and application of the sequence-to-sequence (seq2seq) model 
[8] in machine translation, researchers attempt to apply the 
seq2seq model to generate method names automatically after 
fine-tuning. 

To accurately represent the semantic features of each 
method, we collect name tokens that appear in the code 
text and put them into a sequence in order. Recurrent neural 
network [9] (RNN) is widely used for sequence processing, 
and it can avoid missing information when dealing with long 
sequences. We use it to encode the input token sequence 
and get the semantic context representation. To accurately 
represent the structural features of each method, we propose 
a novel code graph representation to correctly represent the 
structural context of it. Graph neural networks [10] (GNNs) are 
widely used for graph data processing, and it can obtain more 
structural information of the input graph than other models. 
We use it to encode the CRG of each method and get the 
structural context representation. 

Since each token in the input sequence has a different 
degree of importance to the word generated, we used an 
attention mechanism to make the model pay more attention 
to the critical parts of the input sequence. Additionally, out- 
of-vocabulary (OOV) tokens frequently appear in the source 


code, thus we incorporate the copy mechanism employed 
in pointer generator [11] to solve the OOV problem. To 
prove the effectiveness of the proposed model, we conducted 
many experiments on the open-source dataset Java-Small. The 
contributions of this paper are as follows: 


e To the best of our knowledge, we are the first to propose 
to match tokens in the sequence with nodes in the 
code graph to learn the representation of two combined 
features. 

e We propose a new representation to represent the context 
of source code. With the help of the new representation, 
our model can capture the semantic features and the 
structural features of source code well in method name 
generation task. 

e We conduct extensive experiments on the public dataset. 
The results show that our model can increase the accuracy 
of method name generation significantly compared to 
other baseline models. 


II. RELATED WORK 


Proper method names can briefly summarize the key func- 
tions of the method as accurately and concisely as possible. 
The code summary generation task needs to generate the 
sentence to describe the method clearly, which is much similar 
to the method name generation task. Therefore, in this section, 
we will cover both aspects of method name generation and 
code summary generation. 


A. Method Name Generation 


Liu et al. [12] develop a deep learning model to detect 
consistency between method names and method entities. Their 
model constructs two different vector spaces by encoding 
method names and source text separately and placing similar 
method names and entities in adjacent positions of each 
vector space to achieve consistency judgment. For inconsistent 
judgment, method names are suggested through information 
retrieval. Code2vec [5] extracts the paths in the AST of 
the code and represents the code as a weighted sum of 
all path representations, finally producing a code vector for 
each method. As mentioned above, the model in code2vec 
is based on tree structure similarity matching. When the 
model processes new samples, its generalization ability is 
relatively weak. Code2seq [7] solves the shortcoming by using 
the encoder-decoder structure to generate the method name 
sequence, and the input data is still the paths in the AST of 
the code. Allamanis et al. [13] encode the program source 
code using a convolutional neural network (CNN) with an 
attention mechanism and then predict the method name using 
a predictive model. Xu et al. [14] use a hierarchical attention 
model to encode code area blocks and code token sequences, 
and then use a decoder to generate method names with the 
processed data. Fernandes et al. [15] propose a new code 
representation based on the combination of a graph neural 
network (GNN) model and a seq2seq model, and method 
name generation is one of the downstream tasks in their paper. 
Ziigner et al. [16] propose a new model to learn the features 


of source code using language-agnostic features, and method 
name generation task is chosen to prove the effectiveness of 
their approach. Ge et al. [17] apply the keyword extraction 
technique [18] to the combination of a GNN model and a 
seq2seq model to improve the accuracy of the method name 
generation task, and their model’s performance is the best in 
the method name generation task. 


public Preferences putInteger(String key, int val) { 
properties.put(key, Integer.toString(val)); 
return this; 


} 


public Preferences (String key, long val) { 


properties.put(key, Long.toString(val)); 
return this; 


} 


Fig. 1. Example of proper method names 


B. Code Summary Generation 


The code summary generation is much similar to the text 
summarization of Natural Language Processing (NLP). Many 
researchers migrated many strong models to the code summary 
generation task. Iyer et al. [19] first combine long short-term 
memory (LSTM) with the attention mechanism to generate 
its corresponding description according to SQL query state- 
ments. Ahmad et al. [20] propose a transformer-based model 
and generate source code annotations using relative position 
embedding (RPE). Wei et al. [21] regard code summary 
generation and code generation dual tasks and directly use 
the correlation of the two tasks to train the model. The 
above methods only pay attention to the semantic features of 
source code and ignore the structural features of it. Therefore, 
researchers put forward many methods to solve this problem. 
Hu et al. [22] propose a new way of traversing ASTs, which 
retains the structural information of code well in the sequence 
obtained by traversing. LeClair et al. [23] use independent tree- 
based LSTM components and solve the problem of exposure 
bias through reinforcement learning. LeClair et al. [24] encode 
code token sequences and ASTs separately, and this processing 
strategy enables the model to learn the features of token 
sequences and code structures independently. LeClair et al. 
[25] employ a GNN model to encode ASTs and further 
improve the performance of the model. 

The above approaches of method name generation and code 
summary generation have achieved remarkable performance. 
However, these models cannot fully exploit the structural 
information of source code and establish a correlation between 
two features. In this paper, we address these two major issues 
to further improve the performance of the method name 
generation task. 


/** Removes all listeners on this actor. */ 

public void clearListeners () { 
listeners.clear(); 
captureListeners.clear(); 


} 


Fig. 2. Example of proper method summary 


III. APPROACH 


In this paper, the whole task is divided into two main steps. 
In the first step, we process source code to get the token 
sequence and the CRG. In the second step, after encoding 
the code token sequence and the CRG independently, the two 
features are passed into the decoder for generating the target 
method name sequence. The general architecture of our model 
is shown in Figure 3. 

In the encoding phase, our model encodes the token se- 
quence with a Gate Recurrent Unit (GRU) [26] and encodes 
the input CRG with a GNN. To match tokens in the token 
sequence with nodes in the CRG, we select the corresponding 
nodes that appear in the token sequence and sort them ac- 
cording to their order in the sequence. After selecting nodes, 
the model obtains a sequence of tokens and the corresponding 
node sequence. We use these two sequences to represent the 
semantic context and structural context of each method and 
obtain the two representations for later decoding. 

In the decoding stage, we add a decoding attention mecha- 
nism in the decoder architecture to allow the model to select 
the more important parts of the input sequence. Meanwhile, 
the OOV words that appear in the source method text are 
copied directly with the help of a pointer network [11]. 


A. Building Semantic Context 


This paper extracted the method body in advance to extract 
tokens directly from the fields of the method body. These 
tokens are added in a sequence in order after extraction. Con- 
sidering the naming convention in programming, compound 
tokens are not suitable for semantic extraction, and they are 
decomposed into sub-tokens for later use. As shown in Figure 
1, the semantic expression of “putInteger’” is incompatible 
with the semantic expression in natural language, while “put 
integer” clearly expresses the key functionality of method 
accurately. Therefore, “putInteger” is decomposed into “put 
integer” to narrow the gap between program language and 
natural language. These sub-tokens are collected in order into 
the sequence to represent the semantic context of each method. 


B. Building Code Relation Graph 


In this paper, we use the graph output format [27] of code 
analysis tool fastast [28] to construct a CRG for each method 
and represent the structural context with it. CRG is constructed 
in the following steps. 

Names describe syntax nodes in CRGs, and tokens are 
described by strings they represent in them. CRG uses “Child” 
edges to connect nodes according to the AST. Since there is 


no order between tokens, “NextToken” edges are added to 
connect each token to its successors. To capture control and 
data flows in programs, extra edges are added to indicate the 
use and updating of variables. For such a token v, let DË (v) 
be the set of tokens at which the variable could have been 
used last. “LastUse” edges are used for connecting v to all 
elements of D? (v). Additionally, whenever an assignment 
v = expression is observed, v is connected to all variable 
tokens occurring in expression using “ComputeFrom” edges. 
Additionally, return tokens are also connected to the method 
declaration using “ReturnTo” edges. 

In addition, the token nodes are decomposed into sub-token 
nodes to match tokens in the sequence with nodes in the code 
graph. For such a token v, let DÙ (v) ,v € DS (v) be the sub- 
token set generated by decomposition of the token. Element v 
is connected to all elements in D’ (v) with “SubToken” edges. 
For each element in DS (v), “NextToken” edges are logically 
used to connect each child token to its successors. 


C. Semantic Context Representation 


A vector representation needs to be constructed for the text 
content of each method to convert the semantic information 
into vectors. In our approach, the token in the sequence is 
replaced by its word embedding. For example, for semantic 
context information F; = [token1,tokeno,...,token,|, the 
sequence Vr, = [v1, v2,...,Un] is obtained from the token 
embedding layer, in which v; represents the embedding vector 
of the token in the sequence. Because the vector sequence may 
have different lengths, we make the vector sequence have the 
same length by filling the sequence zero vector whose length 
is less than the maximum length. Finally, the semantic context 
of each method can be represented as a vector sequence. 

1) Encoding Token Sequence: Our model encodes the token 
sequence with a GRU. The input sequence Vp, represents the 
semantic context of the method. Each vector v; in the sequence 
represents the token of the method text context. 

For each time step t, the model will select vector v; into 
encoder. The model gets a hidden state vector h; as the 
output in this time step. By collecting all the outputs of each 
time step, the model obtains a list of hidden state vectors 
H; = [hi,h2,..., hn], which is the output of the encoder. 
Eventually, the input sequence Vp, is learned by the encoder 
and converted into a hidden vector H;. The calculation details 
are given as follows: 


he = f (ve, he-1) (1) 
he = g (Ye-1; he-1, H;i) (2) 


Equation 1 is the RNN encoder: h is the hidden state in 
the time step t. The function f represents the RNN dynamic 
function, approximated by a GRU function in this paper. 
Equation 2 is the hidden state of the decoder: h+ is the hidden 
state of the decoder in time step t. The function g represents 
the RNN dynamic function, approximated by a GRU function 
in this paper. 
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Fig. 3. Source code is processed token sequences and CRGs, fed into the different encoders to extract semantic and structural features. Then, the two features 
are combined in order for representing the context of source code and the context representation is learned. Finally, the model uses a generator to generate 


the method name sequence from the learned context representation. 


Finally, the model obtains the output of the encoder H; = 
(hy, h2, ..., hn] and the final hidden state h,, for later use. 


D. Structural Context Representation 


A vector representation needs to be constructed for the 
CRG of each method to convert the structure information 
into vectors. In our approach, the node in the graph is 
replaced by its node embedding. For example, for structural 


context F; = |node;, nodez, .. . , noden], the sequence Vr, = 
vi, V9, ae Un | is obtained from the node embedding layer, 


in which v, represents the embedding vector of nodes in the 
sequence. Similarly, we make the vector sequence have the 
same length by filling the sequence zero vector whose length 
is less than the maximum length. Finally, the structural context 
of each method can be represented as a vector sequence. 

1) Encoding CRG: Unlike natural language, the source 
code contains complex and easily accessible structural infor- 
mation that the CRG can represent. GNNs can be directly 
used to encode the graph data and capture the structural 
features of source code. Different types of edges appear in 
CRGs and we should choose a graph neural network that 


is capable of handling different types of edges. Relational 
graph convolution network (R-GCN) [29] is an improvement 
of graph convolution network (GCN) [30], which can handle 
the influence of different edge relations on nodes in the graph. 
Therefore, R-GCN is used to extract the feature of CRGs in 
our approach. 


Since the messaging between graph nodes is simultaneous, 
the time-step feature in the sequence is not available in the 
graph node hidden state through R-GCN. The gated graph 
sequence neural network (GGNN) [31] introduces time steps 
in the graph message passing process and makes the output 
graph node hidden state have time-step characteristics. After 
encoded by the GGNN, the structural features can cooperate 
well with the semantic features. First, R-GCN is adopted to 
extract the features of CRGs and avoid the cross-influence 
between different types of edges. 


Graph G = (V,E, R) with nodes (entities) v; € V and 
labeled edges (relations) (v;,r, vj) € E, where r € R is 
a relation type. The specific message passing process is as 


follows: 


nit) = wn (1) +4 wE nl ) (3) 


pee 


TER jENT Ci,r 


where M? denotes the set of neighbor indices of node i 
under relation r € R. c;,, is a problem-specific normalization 
constant that can either be learned or chosen in advance (such 
as cir = |). 

To prevent overfitting and reduce model parameters, basis 
decomposition and block diagonal decomposition are intro- 
duced for regularizing the weights of R-GCN layers. 

Then, the GGNN is employed to make the output graph 
node hidden state have time-step characteristics. 

Graph G = (V, E, X) is composed by node set V, edge set 
E and node embedding set X. For each u € V, it corresponds 
to a node embedding X, € R?» , where dp represents the node 
embedding dimension. The specific message passing process 
is as follows: 


e Each node needs to send messages to its neighbors. The 
messa W i 

ge mù’ to be sent by each node is converted to the 

current hidden state h% by function f®. In this paper, 

we use a simple linear layer to represent f) at time step 

t. The model initializes the hidden state pO) of each node 

by the node embedding X,, corresponding to each node. 


m= f(A) (4) 


e Each node u aggregates messages from its neighbors by 
element-wise summation. N (u) represents the neighbor- 
ing nodes of node u. 


MO= SO mo (5) 


vEN (u) 


e Fach node u updates the status of its current time step 
based on the aggregated message MË, and the update 
function f is approximated by a GRU function in this 
paper. 

MEDAR ) 6) 


The procedure of message passing is rolled out for the time- 
step T, and the hidden state of each node u at the final time- 
step Ay) is used as node representation. Finally, the node 
[Pas Basses 


hidden state sequence H, = is obtained, 


m 
where m> 7. 

2) Selecting Nodes: After encoding the token sequence and 
the CRG, we have obtained the semantic context H; and the 
structural context H,. By selecting nodes that appear in the 
token sequence, we update the node hidden state sequence 
H, from hi, hy, -hal to [h h3, i ha] . The execution 


9 m 


procedure is given as follows: 


E. Method Context Representation 


We denote by Rep the context representation of a method 
which can be used in a learning model. Given a method M, its 
semantic context Representation H; and its structural context 


Algorithm 1 Node Selecting Algorithm 
Input: Graph G = (V, E), Node Set V, Edge Set E, Token 
Sequence S+, Node Sequence Sh. 
Output: Node Sequence S,,. 
1: Select Related Nodes From Graph 
2: Graph G = (V,E), Node Set V, Edge Set E, Token 
Sequence S+, Node Sequence Sn 
3: Initialize Sp to empty 
4: for node in V do 
5 if node in V then 
6: Append node to Sn 
7 
8 
9 





end if 
: end for 
: Sort Nodes in Sn by their order in S4 
10: Sn = Sort(S,,) 
11: return Sp 





representation H; , we denote by C Pair the set of all pairs 
of token hidden state vectors and their corresponding node 
hidden state vectors: 
CPair; = Í (hy, ny) | hy € H,,h, € H;} (7) 

where h; represents the semantic context features and h; 
represents the structural context features. 

Finally, We use a sequence of C Pair; (M) to represent the 
method M by collecting them in order. 


Rep (M) = [CPair,,CPairo,..., 
F. Decoding to Generate Method Name 


The decoder is used to convert represent hidden vectors H; 
into target sequences Y = [y1, Y2, . . - , Yml- 


p (yt | Y1; -3 Yt-1, Vri) = $ (Ye—-1, he, Hi) (9) 


where function s is possibility calculation function, approxi- 
mated by softmaz function in this paper. 

1) Attention Mechanism: Our model uses the attention 
mechanism to pay more attention to the important parts of 
the input sequence. The method adopted by the attention 
mechanism is that, at each step of the decoder side, some 
vital part of the encoder side is selected to form the context 
information matrix and then output the results of each step. 

Specifically, for time step t, the decoder receives a hid- 
den state s; for the decoding process. At the same time, 
the decoder will receive the hidden state vector sequence 
H; = [hi,ha,..., hn] from the encoder. The model calculates 
the inner product to get the score vector of attention e; in each 
step. The formula is as follows: 


ei =v" tanh (Wih; + W28) (10) 


where W; € Rx% and Wọ € R%%*42 are the weight 
matrices, and v € R® is the weight vector. 

Then, the possibility calculation function is used to get the 
probability distribution of attention weight at, and the output 
of the attention module a! is obtained by weighted summation 


CPair,| (8) 


of the hidden state vector sequence with at. The calculation 
details are given as follows: 


e = [e1, e2, ..., €n] (11) 

a’ = softmax (et) (12) 
N 

a =X ath; (13) 
i=1 


Finally, [az; s;] is calculated as the output of the decoder by 
joining the attention module output a; with decoder hidden 
state s+, where [;] denotes the concat operation. 

2) Pointer Net: The seq2seq and attention mechanism 
model can generate text freely, but it cannot process OOV 
words. Thus, we incorporate a copy mechanism employed in 
pointer generator [11] to solve the OOV problem. Specifically, 
for each time step t of prediction, a generation probability 
Pgen € [0,1] is dynamically calculated. The formula is as 
follows: 


Pgen = sigmoid (Wi. hš + ws + wi r + bptr) (14) 


where Wp«, Ws, We are learnable parameters, s; is the de- 
coding state, hf is the context vector, x, is the input of the 
decoder in time step t. 

In the decoding phase, a comprehensive dictionary needs to 
be maintained. The comprehensive dictionary is the original 
dictionary with all the OOV words appearing in the source 
added, and the model calculates the probability of all words 
on this comprehensive dictionary. The formula is as follows: 


P(w) = Pgen Pvocab (w) + (1 — Pgen ) 5 at (15) 
noting that if w is an OOV word, then P(w) is zero; similarly, 
if w does not appear in the source document, then } 7... at 
is zero. 

At each decoding step, the decoder outputs a token based 
on the P(w), the loss function is the negative log-likelihood of 
the generated token. Then, the whole encoder-decoder model 
is trained in an end-to-end way. The overall loss for generated 
method names is as follows: 

12 
loss = F 5 — log(P(w)) 
t=0 
IV. EXPERIMENT 


(16) 


The basic idea of our research is to learn the rich semantic 
and structural information of the source code that enables 
generating method names automatically. In short, our study 
mainly focuses on three research questions. We have adapted 
the model structure to the characteristics of the programming 
language and experimented with it to find the most suitable 
way to handle the source code. In short, our study mainly 
focuses on three research questions. 

e RQI: What each part of our model contributes to the 

task? 

e RQ2: How the main parameter settings of the encoder- 

decoder model affect the task? 


e RQ3: What are the advantages of our model compared to 
other models? 


A. Experiment Settings 


In our experiment, we use a 128-dimensional embedding 
vector to represent each token in the token sequence and 
each node in the CRG, and use a 256-dimensional vector to 
represent the hidden state of them. The GGNN is rolled out 
for four steps to get the final representation of nodes in CRGs. 
We remove all the words whose frequencies of occurrence are 
less than five to build the vocabulary and replace them with 
the unknown symbol. We initialize the Adam optimizer [32] 
with a learning rate of 0.00005. 

All the experiments are implemented using the PyTorch 1.8 
framework with Python 3.9, and the experiments were con- 
ducted on HPC with four Nvidia Tesla 100 GPUs with 16 GB 
memory, running CentOS 7.5. More experiment settings about 
our approach can be found in the public code repository!. 








TABLE I 
STATISTICS OF OUR DATASET 
Java-small 
#projects - training 10 
#projects - validation 1 
#projects - test 1 
#examples - training 691K 
#examples - validation 23K 
#examples - test 57K 


Avg. code length - subtokenized (training) 37 
Avg. number of nodes of CRGs (training) 
Avg. target length (training) 3 





B. Dataset 


In this paper, we conducted experiments on java-small [7] 
dataset. This dataset contains eleven Java projects on GitHub. 
This dataset is widely used for method name generation tasks, 
and the evaluation process can be divided into two ways: 


e Train and test across projects, such as Alon et al. [7] and 
Fernand et al. [15] 

e Train and test across files, such as Allamanis et al. [13] 
and Xu et al. [14]. 


If the dataset is divided across files, a file can be in the 
training set, while another file from the same project can 
be in the test set. This makes the task much easier for 
model, because information leakage happens in this way of 
division, and there are often duplicates in different file of 
the same projects. Thus, we decided to split the dataset to 
train/validation/test across projects. 


C. Evaluation Matrix 


Recall-Oriented Understudy for Gisting Evaluation 
(ROUGE) [33] includes measures to automatically determine 
the quality of a generated word sequence by comparing it to 
ideal word sequences created by humans. Since the task can 
be viewed as a form of generation, we report ROUGE-1 and 


"https://github.com/QZH-eng/SGMNG 


ROUGE-L scores, which we believe to be useful indicators 
for the quality of results. ROUGE-N is computed as follows: 


ROUGE — N 
a Di Se (Reterercé) oranes Countmatch (gram) (17) 
set Reference} Dyas Count (gram) 


where n stands for the length of the n-gram, gram, and 
Countmatch (gram, is the maximum number of n-grams 
co-occurring in a candidate summary and a set of reference 
summaries. 

The L in ROUGE-L refers to the longest common subse- 
quence (LCS). A sequence Z = [21, 22,..., Zn] is a subse- 





quence of another sequence X = [21,%2,...,%mJ], if there 
exists a strict increasing sequence [i1,%2,...,%%] of indices of 
X such that for all j = 1,2,...,k, we have xi, = zj. Given 


two sequences X and Y, the LCS of X and Y is a common 
subsequence with maximum length. 


TABLE II 
EXPERIMENTAL EVALUATION RESULTS ON TEST DATASET 











Java-small 
Model ROUGE-1 ROUGEL 

ConvAttention 33.1 — 
Code2seq 43.0 = 
BiLSTM+Copy 42.5 45.6 
GNN+Copy 50.5 48.9 
BiLSTM+GNN+Copy 51.4 50.0 
KG-MNGen 53.4 53.6 
SGMNG 54.09 54.03 


D. Baselines 


We compare our approach with following baselines: 

e ConvAttention. Allamanis et al. [13] generated method 
names with convolutional attention neural networks. 

e Code2seq. Alon et al. [7] extracted paths of each 
method’s AST and select relevant ones through an at- 
tention mechanism to generate the method name. 

e BiLSTM+Copy. Fernandes et al. [15] generated the 
method name by combining the seq2seq model with the 
copy mechanism. 

e GNN+Copy. Fernandes et al. [15] proposed a graph se- 
quence framework to model and extract semantic features 
and structural features of source code for method name 
generation. 

e BiLSTM+GNN+Copy. Fernandes et al. [15] extended 
the seq2seq model with a graph sequence model and 
performed method name generation task in this model. 

e KG-MNGen. Fan Ge et al. [17] decomposed the method 
name generation task into a keyword extraction task and a 
method name generation task and used the dual model to 
improve the model’s performance, which achieves state- 
of-art results on the method name generation task. 

The experimental results for each of the above baseline 
models are shown in Table II. The evaluation results are similar 
to what their original paper reported. Thus, we reuse the 
experiment results presented in their paper. Since our method 


is based on generation, we only compare it to the generation- 
based method to ensure fairness. F1 score in earlier work is 
omitted since it is equivalent to ROUGE-1 score. We note that 
there is no widely accepted metric for this task, and further 
work identifying the most appropriate metric is required. 


TABLE III 
EXPERIMENTAL EVALUATION RESULTS OF EACH STRATEGY 











Java-small 
Mogel ROUGE-1_ _ ROUGE-L 
w/o R-GCN 53.1 53.0 
w/o GGNN 49.7 49.7 
w/o R-GCN+GGNN 39.0 38.9 
w/o Node-Selection 45.0 44.9 
SGMNG 54.09 54.03 


E. RQI: The Effectiveness of Each Strategy 


To better understand the importance of different components 
of our approach, we evaluate the effectiveness of various 
aspects of our model. The experimental results under different 
strategies are shown in Table III. 

Overall, the complete model is superior to all the com- 
parative variants under all the evaluation metrics. Firstly, we 
did experiments to verify that structural information really 
plays an important role in the task. The performance can be 
improved by over 10% when the GNN is employed in the 
model. As shown in Figure 4, compared to sequence-based 
models, our model can not only mine patterns in structural 
information but also capture long dependency properties of 
source code. This observation indicates that the effectiveness 
of the structural information in boosting the accuracy of 
method name generation task. 

Secondly, experiments were designed to verify that the 
employment of the GGNN and the R-GCN can significantly 
improve the performance of the model. Compared to the base 
model w/o R-GCN+GGNN, the GGNN equipped model has 
made significant improvement under the evaluation metrics. 
This is because the GGNN introduces time-step in graph 
message passing process and the time-step is critical in the 
sequence generation task. Compared to the base model w/o R- 
GCN+GGNN, the R-GCN equipped model has made signifi- 
cant improvement under the evaluation metrics. As described 
previously, there are different types of edges in CRGs. There- 
fore, R-GCN is ideal for encoding CRGs and preventing cross- 
influence between different types of edges. Finally, we have 
made some attempts to reduce memory consumption. We try 
to compress and merge all node vectors into a single vector 
and use it to represent the whole CRG like Code2vec. The 
experiment results prove that the model w/o Node-Selection 
does not represent the features of the graph very well in this 
way. This is because the previously retained information is 
lost or incomplete after compression. Thus, the Node-Selection 
strategy is taken to avoid the loss of structural information. 

In summary, all the components in our model are effective 
for the method generation task. We decided to use the complete 
model to conduct follow-up experiments. 


The model generates the method name based on a specific code structure pattern provided by structural information. 





void addToQueue(PathDeletionContext... contexts) { 
cleanupThread.addToQueue(contexts); 


Ground Truth: add to queue 
SGMNG (with structural information): add to queue 
Seq2seq (without structural information): add context 











generated method name reflects the return type. 





private String readString() throws IOException { 
int length; 
try{ 
length = dis.readint(); 
} catch (EOFException eof) { 
return null; 


byte[] bytes = new byte[length]; 
dis.readFully(bytes); 
return new String(bytes, "UTF-8"); 


} 


Ground Truth: read string 
SGMNG (with structural information): read string 
Seq2seq (without structural information): read 














de cake 





COMPLEX_NAME 


COMPLEX_NAME 








Structural information ensures that the model does not ignore the long dependency properties of the code and the 





PARAMETER_LIST 





Fig. 4. Some examples of how structural information works 


F RQ2: The Effects of Different hyperparameters on The 
Results 


We find that the lengths of samples in the dataset differ 
seriously. Thus, we compare the performance of our model 
under different input lengths to find the best setting. The 
detailed results are presented in Figure 5. We use the number 
of tokens to denote the input length. 

As shown in Table IV, the words in the method name 
sequence tend to appear in the location interval [0,100], if 
they are included in the input sequence. To some extent, this 
observation reflects the programming habits of programmers, 
who usually define a series of important variable names at the 
beginning of a method. These important variable names are 
likely to be used in method names to describe the method. 


























TABLE IV 
THE LOCATION OF APPEARANCE IN INPUT SEQUENCE 

Location | Number | Percentage 

0-49 43682 96.2 

50-99 1253 2.76 
100-149 290 0.64 
150-199 86 0.18 
200-249 48 0.11 
250-299 23 0.05 
300-349 8 0.01 

















As the input sequence becomes longer, the model can 
receive more information, but the amount of unimportant 
information received increases in parallel. That is why the 
score gradually increases as the length of the input sequence 
increases, but then gradually decreases after reaching a peak. 
The experiment results prove that the model can obtain the 
most information from the input sequence when the maximum 
input is set at 300. As the maximum input length continues to 


grow, too much unless input information can cause a decrease 
in model performance. 
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Fig. 5. The ROUGE scores under different input lengths 


In the model testing phase, this paper adopts the beam 
search strategy [34] to search for the suitable decoding results. 
As shown in Figure 8, we counted lengths of method name 
sequence in the training dataset. Obviously, the setting of the 
maximum decoding length in the beam search strategy affects 
the experiment results significantly. Experiments with setting 
different decoding lengths are conducted to study the effect 
of maximum decoding length on model performance, and the 
results are shown in Figure 6. 


G. RQ3: The Effectiveness of Our Approach 


Our proposed approach is referred to SGMNG. We com- 
pared SGMNG with several strong baselines under the 
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Fig. 6. The ROUGE scores under different decoding lengths 





200000 4 


150000 4 


100000 4 


50000 4 








Fig. 8. The lengths of method name sequences 


ROUGE- 1 and ROUGE-L evaluation metrics. The evaluation 
results are presented in Table II. 

As shown in the Table I], ConvAttention achieves the 
lowest score among the baselines, which might due to the 
cross-project choice. ConvAttention was originally designed 
to predict method names across files. As previously mentioned, 
information leakage always happens in this way of dividing. 
The graph-based models perform significantly better than 
sequence-based baselines, which indicates that the structural 
information of source code does improve the performance of 
the method name generation task. 

Overall, SGMNG achieves the highest score in the 
ROUGE-1 and ROUGE-L metrics. Compared with the state- 
of-the-art model KG-MNGen, our model achieves better 
performance more effectively. KG-MNGen employs a key- 
word extractor to extract keywords from source code, which 
results in additional costs. Compared with the similar model 
BiLSTM+GNN+Copy, SGMNG improves ROUGE-1 and 
ROUGE-L score about 2.6%-4% respectively. We argue that 
the reason is the new representation enables the model to solve 
the drawbacks of learning semantic and structural features of 
the source code separately. Additionally, due to the adoption 
of the new representation, SGMNG no longer needs to use 


separate attention mechanisms for sequences and graphs, re- 
ducing the number of parameters. 

Additionally, we also studied the need for the use of copy 
mechanisms in models. As shown in Table V, we find that 
55% of the words in the method name sequence appear in the 
input sequence, and 45% of the words in the method name 
sequence never appear in the input sequence. Therefore, the 
method name generation model needs to have the ability to 
generate text abstractly and copy the source text directly. Our 
model uses a GRU to encode the input sequence and decode 
for the target sequence by another GRU, enabling the model 
to generate text abstractly. Moreover, we add PointerNet in the 
model to enable the model to copy the source text directly. 

In summary, our model performs best among several strong 
baselines on the evaluation metrics. 











TABLE V 
THE APPEARANCE OF METHOD NAME WORDS IN TOKEN SEQUENCES 
Appearance | Number | Percentage 
45409 55 
X 37033 45 

















V. CONCLUSION AND OUTLOOK 


In this paper, inspired by the correspondence of tokens 
in source code and nodes in code graph, we propose a 
structure guided method name generation approach. Our ap- 
proach uses pairs of token hidden state and node hidden 
state to represent each method’s context and generates the 
method name with them. Due to the employment of the new 
representation, SGMNG outperforms the similar model BiL- 
STM+GNN+Copy in the method generation task, improving 
ROUGE metrics by 2.6%-4%. Besides, SGMNG can generate 
method names more effectively than the state-of-the-art model 
KG-MNGen without adopting a keyword extractor. In the 
future, we plan to apply our proposed approach to other 
downstream tasks and try to extend it to other programming 
languages. 
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